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Abstract 


The  norm  has  been  widely  studied  as  a  criterion  for  curve 
fitting  problem.  This  paper  presents  an  algorithm  to  solve  discrete 
approximation  problems  In  the  norm  when  the  parameters  are  re¬ 
stricted  by  linear  constraints.  The  algorithm  Is  a  special-purpose 
linear  programming  dual  method  which  employs  a  reduced  basis  and 
■ultlple  pivots.  Results  of  the  computational  experience  with  a 
computer  code  version  of  the  algorithm  are  presented. 
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1.  Introduction  and  Problem  Statement. 

The  lm  norm  problem,  also  called  mini max  and  the  Chebychev 
problem,  has  been  widely  studied  as  a  criterion  for  curve  fitting. 

This  procedure  minimizes  the  maximun  residual,  and  is  particularly 
well  suited  to  problems  found  in  numerical  analysis  (Rabinowitz  (1968)), 
where  a  typical  application  arises  when  values  are  truncated  to  a  fixed 
nunber  of  decimal  places;  and  the  resulting  errors  due  to  round-off 
are  assumed  to  have  an  underlying  uniform  distribution.  Of  particular 
interest  in  mwerical  analysis  is  the  error  between  the  approximation 
of  a  function  and  the  true  function.  It  is  desirable  to  place  a 
bound  on  the  error,  and  in  the  absence  of  a  priori  knowledge  of 
what  the  numbers  involved  will  be,  it  seems  reasonable  to  allow  for 
the  worst  case.  Then  the  goal  is  to  utilize  a  procedure,  for  approx¬ 
imating  the  function,  which  produces  the  miniiman  maximum  error,  hence 
the  minimax  or  norm  method  (see  Ralston  (1965)). 

Although  other  procedures  are  more  popular  for  "statistical 
data  analysis,"  the  norm  may  be  useful  in  identifying  outliers 
(Sposito  (1976)).  Stiefel  (1959)  developed  a  method  called  the 
"exchange  method"  for  finding  norm  estimates,  and  he  later  (1960) 
established  the  equivalence  of  the  method  to  the  simplex  procedure. 
Harter  (1975)  and  Stiefel  (1964)  discuss  the  historical  development 
of  the  Lw  norm  estimation  procedure,  and  Appa  and  Smith  (1973)  iden¬ 
tify  a  manber  of  important  properties. 


It  is  generally  accepted  that  some  form  of  linear  programing 
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algorithm  provides  the  most  efficient  way  to  determine  the  norm 
estimates.  The  purpose  of  this  paper  Is  to  present  a  specialized 
dual  linear  programming  algorithm  for  obtaining  a  Chebychev 
approximation  to  an  overdetermined  system  of  linear  equations  for  the 
model  C  ■  ♦  c  ,  with  linear  constraints  on  the  estimates. 

Analogous  linear  programming  approaches  for  the  linearly  constrained 
Lj  non  problem  are  given  by  Barrodale  and  Roberts  (1973)  and  Arm¬ 
strong  and  Hultz  (1977).  The  algorithm  given  here  uses  a  reduced 
basis,  multiple  pivots,  and  a  reduced  ratio  test.  In  the  absence  of 
constraints  on  the  estimates,  It  reduces  to  our  earlier  unconstrained 
algorithm  (Armstrong  and  Sklar  (1979)).  An  alternative  algorithm  to 
solve  the  unrestricted  Chebychev  problem  Is  given  by  Barrodale  and 
Phillips  (1975). 
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1.1  Statement  of  the  problem 

The  general  l.  nom  problem  may  be  characterized  In  the  follow- 

,  „  ,  .  1.  1  •  1,  2... .«•»«,  represent 

Ing  way.  Let  (c^  ,  a^  •  a^  •  *•••  aim'*  1 

the  values  observed  during  n,  repetitions  of  an  experiment,  where  c 
is  a  vector  of  observed  values  of  the  dependent  variable  and  the  a  s 
are  the  observed  values  of  the  Independent  variables  (the  predictor 
variables).  The  objective  Is  to  determine  estimates 
for  the  parameters  8  ■  (8r  % . Bj  which  solve  the  problem: 

(1)  Minimize  the  value  of 

X  »  max1nM«{|cj  -  *  a12®2  "  ***  "  a1m®«^’ 

1  •  1 ,2* • • • *o 1 

It  Is  a  well-known  result  that  (1)  may  be  expressed  In  a  linear  pro¬ 
gramming  formulation  (L.  M  as  follows: 

(2)  Minimize  X 


subject  to  c,  -  V  <  j  t  ^  .^Sj  I',  ♦  »  •  1  ’  '•* . "I 

uhere  the  opt,™!  est, metes  of  g  »„!  the  devt.tton 


In  matrix  notation,  the  constraints  of  (2)  arr 

(3)  c  -  ex  <  AT8  <  c  ♦  ex  . 

where  e  Is  a  vector  of  ones  and  AT  is  an  n^  by  m  matrix. 

The  additional  linear  constraints  on  the  8  estimates  may  be 


written  as 


d  <  OB  <  u 


where  d  and  u  are  n2-d1mens1onal  vectors  of  lower  and  upper  bounds, 
respectively,  on  the  constraints,  and  Q  is  an  n?  by  m  matrix.  Hot* 


J 
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that  the  vectors  d  and  u  need  not  exist,  so  the  algorithm  to  be  present¬ 
ed  Is  general.  The  constraints  on  the  B's  may  be  appended  to  the 
constraints  of  the  L.  P.  problem  (3),  which  together  may  be  written 
c  -  eX  <_  A^B  <_  c  ♦  eX 
d  £  QB  <  u 

thus, 

C)  ■(><)•  <)•(>• 

which  becomes 

(4)  a  -  lx  <  HT6  <  CU  ♦  eX  , 

*  T 

where  Cl,  CU,  and  e  are  (n^  ♦  n^  ■  n)-d1mens1onal  vectors  and  H  Is  an 

n  by  m  matrix.  It  Is  assigned  that  HT  has  full  colimn  rank;  that  Is, 
rank  (HT)  ■  m.  Rank  deficiencies  can  easily  be  handled  within  the 
linear  programing  framework  (see,  for  Instance,  Ben- Israel  and 
Charges  (I960)),  and  will  not  be  reviewed  here.  Each  Interval  constraint 
may  be  written  as  two  constraints  which,  when  combined  with  the  objec¬ 
tive  function,  results  In  the  following  problem. 

(5)  Minimize  X 

subject  to  HTb  ♦  ex  >  Cl 

HTb  -  eX  <  CU  . 

The  linear  programming  dual  of  (5)  may  be  written  as 

(6)  Maximize  ClV  ♦  CuV 

subject  to  H  *'  ♦  H  »"  •  0 

eTe'  -  eTt"  ■  1 
*'  >  0  ,  »"  <  0  . 


Problem  (6)  may,  of  course,  be  solved  using  the  traditional 
simplex  method,  which  will  be  discussed  In  the  next  section.  This 
discussion  Is  not  meant  to  be  a  complete  description  of  linear  prog¬ 
raming,  but  rather  Is  Intended  to  establish  notation  and  terminology. 

A  short  sixwnary  of  the  simplex  method  follows. 

Given  a  basic  feasible  solution,  corresponding  to  an  extreme 
point  of  the  solution  space,  a  nonbaslc  variable  Is  selected  to  enter 
the  basis.  (In  this  paper  the  usual  terminology  will  be  employed,  where 
a  variable  Is  said  to  enter  (or  leave)  the  basis  when,  In  fact.  It  Is 
the  vector  associated  with  the  variable  which  enters  (or  leaves)  the 
basis,  for  the  vector  space  R**1.)  The  variable  selected  to  enter  the 
basis  Is  a  variable  whose  corresponding  constraint  In  the  dual  of  the 
problem  being  discussed  violates  feasibility.  (In  the  example  of  this 
paper,  for  an  entering  variable  of  problem  (6),  the  corresponding 
constraint  of  problem  (5)  would  be  infeasible.)  The  variable  to  leave 
the  basis  Is  selected  by  determining  the  minimum  of  a  set  of  ratios, 
where  the  leaving  variable  Is  the  first  variable  to  become  Infeasible 
as  the  value  of  the  entering  variable  changes. 

The  new  set  of  basic  variables  differs  by  one  from  the  pro¬ 
ceeding  set,  and  a  new  extreme  point  (Ignoring  degeneracy)  In  the 
solution  space  Is  represented  by  the  basis.  Since  a  finite  number 
of  extreme  points  exist  In  the  solution  space,  and  an  Iteration  as 
described  above  moves  from  one  extreme  point  to  an  adjacent  (Improving) 
extreme  point,  the  algorithm  will  converge  to  an  optimal  solution  In 
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a  finite  rnauber  of  steps.  The  convergence  difficulties  that  arise 
when  degeneracy  Is  present  will  not  be  discussed  In  this  paper.  The 
reader  Is  referred  to  Charnes  (1952)  for  a  discussion  of  the  resolution 
to  the  problem  of  degeneracy.  In  the  next  section,  the  steps  of  the 
general  algorithm  described  here  will  be  discussed  further. 
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2.  General  Linear  Programing  Approach 

Prior  to  describing  the  general  L.  P.  approach  and  the  specific 
algorithm  presented  here,  a  'number  of  terns  are  defined.  (Note  that 
the  omission  of  a  prime  or  double  prime  on  a  term  which  usually  has  one 
or  the  other  Indicates  the  term  applies  to  both  the  prime  and  the  double 

prime  cases.) 

The  first  n1  constraints  of  (5)  are  denoted  as  "variable 
Interval*  (VI)  constraints.  This  terminology  arises  from 
the  fact  that  the  bounds  on  the  Interval  may  be  expanded  by 
Increasing  V. 

The  rest  of  the  constraints  of  (5)  are  the  n^  constraints 
denoted  as  “fixed  Interval*  (FI),  since  the  bounds  on  the 
Interval  may  not  be  expanded. 

^  •  the  number  of  variable  Interval  constraints  In  the 

primal  problem  (5); 

n2  ■  the  mxnber  of  fixed  Interval  constraints  In  (5); 
n  *  ,  the  total  mwber  of  constraints  In  (5); 

H.j  ■  the  m-dlmenslonal  column  vector  for 

the  J-th  column  of  H,  where  H.j  Is  the  same  for 

both  and  ; 

ft.  .  the  (m+1 )-d1mens1onal  augmented  colunn  vector  for 
J 

,  where  a  *1  Is  appended  to  the  vector  H.j  If 
j  <  nj  ,  and  a  0  Is  appended  If  J  * 

H".  •  the  augmented  vector  for  »  where  a  -1  Is  appended 

If  J  <  i»j  ,  and  a  0  Is  appended  If  J  >  . 
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k(1)  ■  the  Index  for  the  1-th  basic  variable,  1  ■  1,2,..., 

m+1.  That  Is,  If  either  or  Is  basic  at  the  1-th 
position,  k(  1 )  ■  r'; 

F  -the  (m*l)  by  (m*l)  basis  for  the  dual  problem  (6)  ; 

b  *  the  original  right  hand  side  for  the  dual  problem  (6), 

b  -  (0.  0,...,0,1)T  ; 

b  •  the  (m*l )-d1mens1onal  updated  right  hand  side  of 
(6).  b  •  F‘]b  ; 

Cp  ■  an  (m+1). dimensional  vector  of  the  original  objec¬ 
tive  function  coefficients  for  the  basic  variables 
of  problem  (6)  ; 

Clj  *  the  original  objective  function  coefficient  In  the 
dual  problem  (6)  for  ; 

CUj  •  the  original  objective  function  coefficient  for  ; 

cj  »  the  reduced  cost  for  ; 

c*  •  the  reduced  cost  for  ; 

y'.j  ■  the  representation  of  H.'j  In  terms  of  the  basis  F, 
so  that  y’.j  •  F’^H'.j  ; 

A  A 

/tj  ■  the  representation  of  H".j  ,  y".j  ■  F  H’tj  ; 

NB*  ■  { J | Is  nonbaslc)  ,  the  Index  set  for  nonbaslc 
variables  of  the  type  w'  ; 

NB"  ■  (j  |iij  Is  nonbaslc) 

NB  •  NB'UNB" 

X  -the  value  of  the  objective  function; 

B  •  the  parameters  to  estimate  In  the  primal  problem  (5). 
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2.1  Steps  of  the  general  L.  P.  simplex  method. 

Given  a  basic  feasible  solution,  the  steps  of  the  general  L.  P. 

simplex  problem  may  be  stated  as  follows. 

Step  1.  Compute  the  reduced  costs  for  nonbaslc  variables; 

a.  C‘  -  Clj  -  ,  for  xj  ,  jtNB* 

b.  •  CUj  -  ,  for  xj  ,  jcNB" 

c.  select  the  variable  to  enter  corresponding  to 
max  {cj  ,  cj  >  °  ;  -cj  ,  Cj  <  0) 

jcNB 

d.  If  cj  <  0  and  cj  >  0  for  all  j,  terminate  with  the 
optimal  solution; 

Step  2.  Compute  the  representation  for  x$,  the  entering  variable 

a.  y‘  ■  F"1H'  for  x'  entering  the  basis  ; 

b.  y"  ■  F'lH“  for  x"  entering  the  basis  ; 

•  S  •  S  8 

Step  3.  Compute  the  updated  right  hand  side 

e  -1 

a.  b  ■  F  'b 

Step  4.  Select  as  the  variable  to  leave  the  basis  that  variable 
corresponding  to  the  minimal)  ratio 

a.  If  x*  Is  the  entering  variable, 

s 


b.  If  x“  Is  the  entering  variable, 

min  I-  ^-j  •  h-  <  0  *  J  "  1f  2 . "fl 

(  yJs  )  yJs 

c.  If  the  minimum  occurs  for  j  •  r  ,  xfc  leaves  the  basis 
Step  S.  Update  F_1  ,  cJf-1  ,  k(r),  NB'  and  NB".  Go  to  Step  1. 
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3.  Reduced  Basis  Structure 

This  section  discusses  how  the  full  tableau  data  may  be  repre¬ 
sented  with  a  reduced  basis.  The  traditional  approach  of  the  prior 
section  for  solving  the  1.  P.  problem  (6)  may  be  altered  In  the  follow¬ 
ing  ways.  Instead  of  using  the  (m+1)  by  (»+l)  matrix  F  to  solve  the 
system  of  equations,  an  »  by  m  basis  may  be  Isolated  to  perform 
the  usual  simplex  steps.  The  primary  purpose  of  this  paper  Is  to  show 
how  the  reduced  basis  may  be  efficiently  used  to  solve  (6). 

Throughout  the  paper,  a  distinction  will  be  made  between  two 
problem  structures:  the  full  structure  which  has  (m*l)  basic  variables, 
this  structure  corresponds  to  the  regular  simplex  tableau;  and  the  re¬ 
duced  structure  with  m  basic  variables,  which  are  the  first  m  of  the 
basic  variables  In  the  full  structure.  Consequently,  there  are  two 
representations  for  a  nonbaslc  variable.  One  representation  is  in  terms 
of  the  (m*l)  basic  variables  of  the  full  structure  and  the  other  Is  In 
terms  of  the  m  basic  variables  of  the  reduced  structure. 

An  additional  distinction  is  necessary  between  the  variable 
Interval  constraints  and  the  fixed  Interval  constraints  of  the  primal 
problem  (5).  In  the  dual  problem  (6),  there  are  n  (or  nj  ♦  nj)  variables 
of  each  type,  *'  and  »"  ,  and  some  operations  will  be  performed  differ¬ 
ently  depending  on  whether  a  variable  relates  to  the  VI  or  the  FI 
constraints.  For  convenience.  It  will  be  assumed  that  the  first  2n^ 
variables  (n^  of  *'  and  n^  of  *")  relate  to  the  VI  constraints  and  the 
remaining  2n£  variables  correspond  to  the  FI  constraints. 

Several  advantages  become  evident  when  the  reduced  basis 
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3.1  Partitioning 

The  proposal  Is  to  construct  a  reduced  basis  with  which  to  perform 
the  simplex  operations.  Since  the  basis.  F,  for  the  full  structure  (6) 
has  linearly  Independent  columns.  It  may  be  partitioned  In  the  follow¬ 
ing  manner: 

6  •  the  m  by  m  basis  for  the  reduced  structure.  The  columns  of 
B  are  those  columns  of  H  corresponding  to  the  first  m  basic 
variables  of  the  full  structure; 

G  ■  the  m-dimenslonal  colimn  vector  of  H  corresponding  to  the 
(m*l)-st  basic  variable  In  the  full  structure,  so  that 
G  •  H 

•k(«*l) 

0  ■  the  row  vector  for  the  (m*1)-st  row  of  F,  so  that 

J1  *k(i)  *S  bas1c  and  k  (i)  -  nl 

-1  If  *'(i)  Is  basic  and  k  (1)  < 

0  If  either  *^(i)  or  Is  basic  and  Mi)  >  • 

for  1  1  1  <  m  ; 

f  •  the  (m*l)-st  element  of  the  column  H  associated 

with  the  (nHl)-st  basic  variable  In  the  full  structure, 
so  that 

If  ,,  I*  t>«1c  «n<l  k  (■♦l)  <  n. 

k (m*l)  » 

If  «"  Is  basic  and  k<»n)  <  n. 
k(m*l)  » 

If  either  **(1|4l)  or  1s  basic  and  k(m*l)  >  n^  . 

Note  that  appending  0^  to  B.^  gives  . *•  and 

appending  f  to  G  gives  H  k(lKl,* 


Then 


where  B  1$  »  by  ».  0  Is  1  by  ■>.  G  Is  ■  by  1 .  and  f  Is  a  scalar. 

ut  r'.(j  ;)  .  wir'.i. 

Then. 

(: !)(: :)  -(10  • 

The  components  of  F  ^  may  be  written  as 

M  •  (B  -  Gf " 1 0)"1  N  ■  -MGf*1  ■  *(B  -  6f  ^0)  *6f 

l  •  -f_10H  q  •  F*1  -  f  10N, 
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3.2  Updated  Right  Hand  Side 

The  right  hand  side  values  for  the  basic  variables  In  the  full 
structure  may  be  described  as  follows: 

bj  ■  the  value  for*k(1),  *  “  ^  ** . ("*1)  , 

*  -1 
where  b  ■  F  b. 

To  facilitate  the  partitioning,  b  may  be  separated  «*>"(#)•  where 
F  Is  the  first  m  entries  of  b  and  W  Is  the  («*l)-st  entry  of  b. 

The  goal  Is  to  write  b  ■  (F  W)T  In  terms  of  the  reduced  basis  B.  It 

«  a  A 

has  been  previously  noted  that  F  b  •  b,  so  Fb  •  b  .  Then,  using 
the  partitioning  of  F  from  the  previous  section,  and  the  separation 
of  b  Into  F  and  W  , 


(S  ?)  (5)  *  (?) 


*  b  . 


Performing  the  multiplication, 

BF  ♦  GM  •  Q 

Ob  ♦  fW  •  1  . 

However,  G  •  ,  so  that 


B^G  ■  B_1H 


•k(»*l> 


where  a  Is  defined  to  be  the  m-dlmenslonal  representation  for 

*k(m+l) 

v  In  terms  of  the  reduced  basis  B.  For  notatlonal  convenl- 

H(ml) 

ence,  let 


R  -  a 

« 

To  continue. 


•h  (»♦!) 


BF  •  GW  •  0  * 

F  ■  B^GW  »  -B'^H 


F  •  -RM. 


•k(«H-l)W  ’  "*  .k(«Ki)W 
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Also, 

Dt>  ♦  fV  •  1  -  D(-RW)  ♦  fU  •  1  -  H(f  -  OR)  •  1  . 

For  notations  convenience,  define 
sumr  ■  (f  -  OR)  , 

so  W(f  -  DR)  ■  1  ,  W(sunr)  ■  1  , 

thus  W  ■  1/stfwr  . 

To  summarize,  the  right  hand  side  values  for  the  basic  variables, 
written  in  terms  of  the  reduced  basis  B,  are 

(s)  ■  (-?)  •  te) . 
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3.3  Selection  of  the  Entering  Variable 

The  reduced  cost,  cj  or  .  for  a  variable  may  be  described  as 
the  change  In  the  objective  function  value  per  unit  change  In  the  value 
of  the  entering  vanable.  The  convergence  of  the  simplex  algorithm 
depends  only  on  selecting  a  variable  to  enter  the  basis  which  has  a 
positive  rate  of  change;  however,  the  Implementation  used  here  selects 
as  the  entering  variable  that  variable  which  has  the  largest  rate  of 
change. 

The  reduced  costs  may  be  computed  by 

cj  •  cij-cJr’S'.j 
*  «J  *  cTFr'fi,4  • 

and  the  largest  rate  of  change  corresponds  to 


$8  |'j  •  cj  >  0  ;  -cj  .  cj  <  0  |  . 

The  optimality  conditions  are  cj  <  0  and  c J  >  0  ,  for  every  jcUB, 
In  which  case  the  algorithm  terminates  with  the  optimal  solution.  The 
optimality  conditions  for  this  dual  problem  (6)  are  related  to  the 
feasibility  conditions  for  the  primal  problem  (5)  ,  In  the  sense  that 
selecting  the  entering  variable  with  the  largest  rate  of  change  In 
X  Is  tantamount  to  bringing  feasibility  to  the  primal  constraint  which 
Is  most  Infeasible  with  the  current  solution. 
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3.4  Representation  of  the  Entering  Variable 

Suppose  the  method  of  the  previous  section  has  selected  ns  as 
the  entering  variable.  The  following  Introduces  additional  notation, 
a  ■  the  first  m  elements  In  the  representation  for  f  ,  which 
Is  y  In  terms  of  the  full  basis  F  ; 

*5 

V  ■  the  (m*l)-st  element  In  the  representation  for  In 
terms  of  the  full  basis  F  ; 

S  »  a  •  the  m-dlmenslonal  representation  for  *  In  terms  of  the 
•  S  * 

reduced  basis  8,  so  that  B’^H  ■  a  . 

•  5  •  a 

Since  the  augmented  column  vectors  for  *'  and  w"  ,  ft*  and  H"  ,  differ 

only  In  the  last  element,  which  Is  defined  to  be  g  ,  then  either  y* 

•  * 

or  y"  (the  representations  In  terms  of  the  full  basis  F  for  w'  and 
• »  s 

<)  may  be  partitioned  as  (a  V)  ,  where  the  sign  of  g  determines 
which  of  H'  or  H"  Is  being  represented. 

Then  using  the  partitioned  version  of  F,  and  separating  y 

T  •* 

Into  ( *  s  V )  ,  we  have 

<:  :hv)  ■(:-) . 

so  that  the  element  g  may  be  defined  as 

!1  If  i*‘  Is  entering  the  basis  and  s  <_  nj 
-1  If  Is  entering  the  basis  and  s  <_  nj 
0  If  either  or  Is  entering  and  s  >  nj 


17 


structure  Is  used.  The  usual  simplex  steps  may  be  performed  In  terms 
of  the  reduced  basis,  and  a  reduced  minimum  ratio  procedure  Is  pre¬ 
sented  which  provides  a  slmpller  ratio  formulation,  a>td  which  may  result 
In  fewer  ratios  computed  than  In  the  full  tableau.  Multiple  pivots  may 
be  performed,  where  a  multiple  pivot  allows  a  movement  to  an  adjacent 
extreme  point  without  updating  the  basis. 
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Performing  the  multiplication, 

Ba  ♦  GV  »  H  e 
Da  .  ♦  fV  ■  g  . 

•9 

A 

Solving  the  first  equation  for  a 


•s  • 


a  •  B-1 (H  -  GV)  •  B_1H  -  B_1GV 

•S  *5  •» 

a  c  •  B_1H  -  B_1H  .V 

•  S  »S  •  k(n*el) 

A  mm  M 

a  ■  a  -  a  V  . 

•s  s  k(*«-l) 

For  simplicity,  let 

S  ■  a  .  and  recall  R  ■  a 


.s 


k (»♦!) 


so  a  *  S  -  RV  . 

Solving  the  second  of  the  equations. 

Da  ♦  fV  ■  g 
0(S  -  RV)  ♦  fV  ■  g 

DS  ♦  V(f  -  DR)  •  g  . 

Recalling  that  W  -  l/(f  -  DR)  •  1/sumr, 
then  OS  ♦  V/W  ■  g 
V/W  ■  g  -  OS  . 

Oeflne  suns  *  g  -  DS  , 
then  V/W  •  suns  , 

V  •  suns/sunr  . 

Therefore,  the  representation  for  In  terms  of  the  full  structure 
Is  y  -  (a  .  V)  T,  which  has  now  been  written  In  terrs  of  the  reduced 

'  #5 

basis,  so  that 

(H  . 
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4.  Simplification  with  the  reduced  structure 

Solving  the  dual  problem  (6)  using  the  reduced  basis  B  offers 
several  advantages,  which  will  be  discussed  In  this  section. 

The  original  primal  problem  (4)  had  n^  ♦  n2  ■  n  interval 
constraints  which  were  separated  Into  2n  constraints,  so  the  dual 
problem  (6)  has  2n  structural  variables.  However,  there  are  n  pairs 
of  variables,  nj  and  x"  ,  for  which  the  H.j  vector  Is  Identical,  and 
the  augmented  vectors  of  H.j  ,  H!^  and  H'.’j  ,  differ  only  In  the  entry 
In  the  (m*l)-st  (last)  element.  To  exploit  this  pairing,  the  algorithm 
computes  reduced  costs  for  n-(m+l)  rather  than  for  2n-{nH-l)  nonbaslc 
variables.  For  any  pair  and  where  j  <  nj  ,  the  reduced  costs 
differ  by  2A  ,  since  H'.j  and  HTj  differ  only  In  the  sign  of  the  "1" 

In  the  last  entry.  For  any  pair  and  it”  where  J  >  n^  ,  the  reduced 
costs  differ  by  CUj  -  CL  j  ,  since  H'.j  and  H".j  are  Identical,  with  the 
last  entry  being  a  zero. 

Additionally,  selection  of  the  variable  to  leave  the  basis  may 
be  simplified  using  the  reduced  structure.  The  minimum  of  ratios  of 
the  form  Rj/S^  win  designate  the  leaving  variable,  and  some  of  the 
ratios  may  be  Ignored,  since  It  may  be  anticipated  they  will  not  be 
the  minimum. 

Also,  a  multiple  pivot  may  be  performed,  which  Includes  a 
"pivot"  where  the  basis  Is  not  changed  In  the  reduced  structure 
(although  In  the  full  structure  a  basis  change  would  occur).  This 
exchange  of  variables  does  not  affect  the  basis  and  therefore  re¬ 
sults  In  a  savings  In  computation. 
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The  reduced  structure  has  m  basic  variables  but  the  (m+l)-st 
basic  variable  In  the  full  structure,  wl^  be  treated  as  a 

pseudo-basic  variable  In  the  reduced  structure.  For  convenience,  the 
variable  will  be  denoted  by  *.  ,  In  the  reduced  structure  just  as 
It  Is  In  the  full  structure.  This  variable  will  play  a  role  In  the 
reduced  structure  as  though  It  were  basic,  In  the  sense  that  a  variable, 
entering  the  basis  In  the  reduced  structure  will  replace  a  current 
basic  variable  If  possible,  and  otherwise  will  replace 
t  Is  replaced  by  *  ,  no  basis  change  occurs  In  the  reduced 

structure,  although  In  the  full  structure  an  explicit  basis  change 
would  occur.  Treating  the  variable  as  a  pseudo-basic 

variable  In  this  fashion  enables  the  sane  extreme  point  path  to  be 
followed  In  the  reduced  structure  as  Is  followed  In  the  full  structure. 


4.1  Computing  the  Reduced  Costs 

Using  the  reduced  structure  may  result  In  some  simplification 
when  determining  the  reduced  costs  of  the  dual  problem  (6).  To  see  this, 
we  consider  the  primal  problem  (5).  In  essence,  the  procedure  will  be 
to  compare  the  1-th  residual  from  the  fitted  plane  (for  the  current 
values  of  0)  to  the  current  value  of  X.  This  Is  analagcus  to  computing. 
In  the  primal  problem  (5),  the  amount  of  Infeaslblllty  of  a  particular 
constraint. 

To  aid  In  the  discussion  of  the  computation  of  the  reduced 
costs,  the  following  terms  are  defined. 

Cj  Is  the  reduced  cost  for  the  j-th  nonbaslc  variable, 
which  may  correspond  to  either  or  ,  for  J  <.  , 

depending  on  the  sign  of  the  residual; 
cj  Is  the  reduced  cost  for  ,  for  J  >  n^  ; 

Cj  Is  the  reduced  cost  for  *“  ,  for  J  >  n^  ; 

hj  •  Cj  -  for  j  <  nj  ,  so  that  hj  Is  the  j-th 

^  ^  A 

residual  from  the  fitted  line  a6  •  (a^&^  ♦  *£1  ♦  ... 

♦  v«>- 

then, 

a.  Cj  »  | c j  -  8TH.j|-X  ,  for  J  <  n^, 

Cj  -  |  hj  |-A  ,  for  J  <  nr 

If  hj  >  0  ,  Cj  Is  the  reduced  cost  for  ; 

If  hj  <  0  ,-Cj  Is  the  reduced  cost  for  » ^  ; 

1.  suppose  hj  >  0,  so  only  ej  will  be  a  candidate.  This 
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means  that  the  point  (a  jgj  .a j j  . .  .ayj) 

on  the  fitted  plane  (which  will  be  called  the  fit)  Is 
below  the  observed  point  (Cj^j^j,...,^).  If 
hj  -  X  >  0,  then  the  fit  Is  below  the  bound  Cj  -  X; 
thus,  the  primal  constraint  Cj  -  X  <  HT0  Is  Infeasible, 
and  *j  Is  the  dual  variable  associated  with  this 
constraint.  If  hj  -  X  <0,  the  fit  Is  within  the 
Interval  bounds  and  the  constraint  Is  feasible. 

2.  For  the  case  where  hj  *  cj  '  8TH.j  <®»  similar 

reasoning  suggests  that  Cj  is  the  reduced  cost  for  *  j , 
and  the  reduced  cost  for  sj  may  be  Ignored,  as  Its 
associated  constraint  In  (2)  must  be  satisfied. 

b.  cj  ■  CL j  -  B^H.j  ,  for  J  >  nj  . 

If  cj  >  0  ,  then  irj  Is  a  candidate  to  enter  the  basis. 

c.  cj  *  CUj  -  B  H.j  ,  for  J  >  nj  . 

If  cj  <  0  ,  irj  Is  a  candidate. 

Then,  the  variable  selected  to  enter  the  basis  is  the  variable  corres¬ 
ponding  to 

max  (Cj  for  c \  >  0  ;  c j  for  cj  >  0  ;  -c^  for  Cj  <  0) 

JeNB  TINS’*  J  J  J  J 

and  terminate  with  the  optimal  solution  If  no  maximum  exists. 

(In  relating  the  reduced  costs  described  above  to  these  encountered  In 
the  full  tableau,  the  sign  of  Cj,  when  Cj  refers  to  xj  .will  be  nega¬ 
tive  In  the  tableau  but  positive  In  this  algorithm.) 

Note  that  when  x  Is  basic  (either  x'  or  it"),  the  reduced 
P  P  P 

cost  for  Its  nonbaslc  complement*  (either  x"  or  s')  Is  not  considered. 

P  P  P 


This  will  be  discussed  In  detail  In  the  multiple  pivot  section,  where 
the  procedure  Is  given  which  determines  when  the  complement  of  a  basic 
variable  should  be  considered  a  candidate  to  enter  the  basis. 
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4.2  Ratios  Using  the  Reduced  Basis 

In  this  section  it  will  be  shown  that  utilizing  the  reduced 
basis  may  result  In  the  computation  of  fewer  ratios,  and  that  the  ratio 
test  procedure  of  the  reduced  structure  (computing  ratios  of  the  form 
R./S^  Is  equivalent  to  the  procedure  of  the  full  structure.  In  the 
sense  that  both  procedures  select  the  same  variable  to  leave  the  basis. 

Prior  to  the  discussion  of  the  ratios.  It  may  be  helpful  to 
summarize  some  of  the  previous  results.  The  following  Is  a  partial 
tableau  for  the  full  structure,  where  the  entries  are  written  In  terms 
of  the  reduced  basis. 


Basis 

> 

RHS 

Ratios 

X 

*k(l) 

s 

srRiv 

X 

-R|/sunr 

(-Rj/sumrJ/tSj-R^sums/sumr)) 

\(2> 

• 

w 

• 

-R^/sumr 

• 

(-R2/sianr)/(S2-R2(sums/sumr)) 

• 

• 

• 

• 

Wk(ml 

• 

• 

w 

• 

-Rpj/sunr 

• 

( -R^/sianr)/ { S^-R^t  sums/sunr ) ) 

*k  (m+1) 

V-  sums 

simir 

1/svanr 

1/sums 

sums  f  0 

The 

1-th  ratio  In  the 

reduced  structure  Is  of  the  form  R^/S^  , 

and  If  Rt/St  Is  the  minimum  ratio  In  the  reduced  structure,  then  the 
corresponding  variable,  w  ,  Is  selected  to  leave  the  basis. 

Two  variables,  and  .will  be  said  to  be  the  same  type  of 
variable  If  both  are  restricted  to  be  >  0  (that  Is,  and  vp  or  If 
both  are  restricted  to  be  <  0  (such  as  or  »“).  The  following  function 
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Is  defined  to  determine  the  sign  of  a  quantity 


sign  (c) 


1  If  £  >  0 
-1  If  C  <0 


The  above  sign  function  will  be  used  to  determine  which  of 
the  ratios,  /S^  ,  should  be  candidates  and  which  may  be  Ignored.  (For 
convenience,  the  following  two  statements  will  be  considered  equivalent: 
(1)  a  ratio  will  be  considered  a  candidate...;  and  (2)  the  variable 
associated  with  a  ratio  will  be  considered  a  candidate...). 

Recall  the  scalar  f  Is  the  (m*l)-st  element  In  the  vector 


H 


and  f  Is  either  ♦!  (for  *‘k(|#+l)),  or  -1  (for  *?k(lHl)).  °r 


0.  When  f  Is  0,  the  type  of  variable  Is  not  obvious,  so  another  function, 

A 

f.  Is  Introduced  to  provide  this  Information.  The  type  of  variable 

A 

(»'  or  w")  associated  with  may  be  known  from 

(  1  If 


Is  basic 


f  • 


k(m*l) 

*'  *f  •£(»,,  "to"' 


for  similar  reasons,  a  function  g  Is  introduced,  which  corresponds  to 

the  entering  variable  : 

„  (  1  If  n*  is  entering  the  basis 

9  •  {  * 

(-1  If  Is  entering  the  basis 

Then,  the  1-th  ratio  In  the  reduced  problem,  R^/S^  ,  will  be  considered 
a  candidate  when 


fg  -  -slgn(Rj)  *  s1gn(S1 )  ,  Sj  t  0  . 

If  this  condition  fails,  the  associated  ratio  need  not  be  considered 
(nor  computed),  since  the  ratio  will  not  be  the  mlnlmun.  In  the  com¬ 
puter  code,  an  equivalent  condition  must  be  satisfied  for  a  ratio  to 

be  considered  a  candidate:  „  „ 

0tg  ■  sign  (Sj)  , 


where  6^  is  defined 


1  if  n;(1)  is  basic 
-1  if  *£(i)  is  basic 


(1  <  1  <  «) 


For  convenience  in  isolating  terms,  the  ratio 

(-Rj/sunr)/(Sj-RjSums/sumr) 

will  be  written  as 

l/((-$f/Rf(sunr  ♦  sums)),  Rj  M  . 

When  Rj  •  0,  the  results  which  follow  concerning  the  relationship 

between  the  reduced  and  full  ratios  follow  in  a  trivial  manner. 

To  show  that  the  reduced  ratio  test  is  equivalent  to  the 

regular  ratio  test  in  the  full  structure,  four  lemmas  will  be  presented. 

For  each  lenvna,  only  the  case  where  1*  *n  the  basis  and  ujj  is 

the  entering  variable  will  be  considered.  The  other  three  cases 

follow  similar  logic.  Since  e*  }  is  basic,  and  variables  of  the 

type  »'  are  restricted  to  be  nonnegative,  then  sumr  >  0  ,  because 

*•  «  1/sunr  >  0  .  Since  ir"  Is  the  entering  variable,  only  vari- 

k  (■*♦  I)  a 

ables  associated  with  negative  ratios  will  be  candidates  to  leave  the 
basis.  Further,  because  w'  and  »*  are  different  types,  the  1-th 

II  l  1 1  I 

ratio  in  the  reduced  structure  will  be  a  candidate  only  when  the  signs 


of  Rj  and  are  the  same. 

Lemma  1 

If  the  1-th  ratio  In  the  reduced  structure  is  a  candidate, 
and  the  corresponding  i-th  ratio  in  the  full  structure  is  a  candidate, 
then  the  (m*l)-st  ratio  is  not  the  minimum  ratio  for  the  full  structure. 
Proof 

If  the  (m*l)-st  ratio  in  the  full  structure  Is  not  a  candidate 
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the  lenma  Is  satisfied.  If  the  (m+l)-st  ratio  Is  a  candidate,  sums  Is 
negative,  since  the  (m*1)-st  ratio  Is  1/sums,  and  only  negative  ratios 
may  be  considered.  If  the  1-th  ratio  In  the  reduced  structure  Is  a 
candidate,  the  signs  of  R^  and  are  the  same.  Thai  the  corresponding 
1-th  ratio  of  the  full  structure  Is  related  to  the  (m*l)-st  ratio  of 
the  full  structure  by 


since  both  terms  In  the  denominator  of  the  1-th  ratio  are  negative. 

This  result  establishes  lemma  1. 

Lemma  2 

If  the  1-th  ratio  In  the  reduced  structure,  R^/S^  i  Is  not  a 
candidate,  but  the  corresponding  i-th  ratio  In  the  full  structure  Is  a 
candidate,  then  the  1 - th  ratio  in  the  full  structure  is  not  the  minimum 
ratio  In  the  full  structure. 

Proof 

Since  the  1-th  reduced  ratio  Is  not  a  candidate,  the  signs  of 
Rj  and  are  different,  which  means  the  first  term  in  the  denominator 
of 


Is  positive,  and  this  requires  sums  to  be  negative  for  the  1-th  full 
ratio  to  be  a  candidate.  Because  suns  Is  negative,  the  («*l)-st  full 
ratio  Is  a  candidate,  so  that  the  1-th  and  (m+1)-st  ratios  In  the  full 
problem  are  related  by 

.  R1  f  0  . 

This  proves  lemna  2. 
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LWM  3 

If  the  problem  is  bounded  In  the  linear  programing  sense 
that  the  values  of  the  variables  have  a  finite  bound,  and  if  the 
i-th  ratio  in  the  reduced  structure  is  a  candidate  but  the  i-th 
ratio  In  the  full  structure  is  not,  then  the  1-th  ratio  in  the 
reduced  structure  is  not  the  minimum  of  the  reduced  ratios. 

Proof 

If  the  i-th  reduced  ratio  is  a  candidate,  the  signs  of 
and  Sj  are  the  same  and  Sj  f  0.  For  the  1-th  ratio  in  the  full 
structure  to  not  be  a  candidate,  the  ratio  must  be  positive,  so 
that 

(^^'^sUrTsiis  >  0  *  R1  *  0  • 

The  first  term  In  the  denominator  is  negative,  which  requires  tfoat 
subs  >  0  ,  so  that 

(Sj/Rj)  sunr  <  suns,  and 

(R1/S1 )  >  (sumr/sums). 

Since  the  1-th  reduced  ratio  is  a  candidate,  and  the  1-th  full  ratio 
is  not,  the  (m+l)-st  ratio,  1/sums,  cannot  be  a  candidate  because 
only  negative  ratios  may  be  eligible,  and  sums  Is  required  to  be 
positive  to  Insure  the  1-th  full  ratio  is  not  a  candidate.  There¬ 
fore,  since  the  1-th  and  (nHi)-st  ratios  are  not  candidates  in  the 
full  structure,  some  other  ratio  In  the  full  structure,  call  It  the 
J-th  ratio,  must  be  a  candidate.  Then 

Mj/fysiAr ' T  sums  <  RJ  '  <>• 


Since  suns  >0,  the  first  term  in  the  denominator  must  be  negative, 
which  requires  that  Rj  and  Sj  have  the  same  signs,  which  means  the 
j-th  reduced  ratio  Is  a  candidate.  Because  the  j-th  full  ratio 
must  be  negative  (for  it  to  be  a  candidate), 

|(-Sj/Rj)sumr|  >  sums  , 

(Sj/Rj)sumr  >  sums  , 

(Rj/Sj)  <  (sums/sumr)  ,  Rj  ,  Sj  ,  Simir  f  0. 

Thus  the  relationship  between  the  i-th  and  j-th  ratios  of  the  full 
structure  Is 

(Rj/Sj)  <  (sums/sunr)  <  (R|/S| ) 

and  this  result  proves  the  lemma.  In  the  event  that  the  problem  Is 
unbounded,  lemma  3  Is  not  satisfied,  and  the  algorithm  terminates. 
Lemma  4 

Suppose  the  1-th  and  j-th  ratios  In  the  reduced  structure 
are  candidates,  and  the  corresponding  i-th  and  J-th  ratios  In  the 
full  structure  are  candidates.  If  the  absolute  value  of  the  1-th 
reduced  ratio  Is  smaller  than  the  absolute  value  of  the  J-th  re¬ 
duced  ratio,  then  the  same  relation  holds  In  the  full  structure. 
Proof 

Let  | R1/S1 |  <  | Rj/Sj |,  which  means  for  the  case  being  con¬ 
sidered  that  Rf/St  <  Rj/Sj ,  since  the  ratios  are  positive.  For  the 
full  structure,  the  corresponding  1-th  and  j-th  ratios  are 


Sunr  is  positive  since  *’  ,  ■  1/sumr  is  basic.  Since  both  the 

1-th  and  j-th  reduced  ratios  are  candidates,  the  first  term  in  the 
denominator  of  both  full  ratios  is  negative  (the  signs  of  R^  and 
are  the  same).  Both  full  ratios  are  negative,  since  both  are  can¬ 
didates,  and  it  is  which  is  entering  the  basis.  Therefore,  for 
each  full  ratio,  either  sums  Is  negative,  or  sums  is  positive,  but 
less  than  the  absolute  value  of  the  first  term  in  the  denominator. 
In  either  case,  sums  may  be  Ignored. 

Since  R1/S1  <  Rj/Sj  ,  then  S</Ri  >  Sj/Rj  ,  so  that 

|  (-Sj/Rj  )sumr  |  >  |  ( -S j/R j ) simnr |  , 
which  means  that 


and  therefore  the  i-th  and  J-th  full  ratios  are  related  by 


which  proves  the  lemna. 

Theorem 

The  ratio  test  procedure  described  for  the  reduced  struc¬ 
ture  is  equivalent  to  the  ratio  test  procedure  for  the  full  struc¬ 
ture,  In  the  sense  that  both  procedures  will  select  the  same  vari¬ 
able  to  leave  the  basis. 

Proof 

It  follows  from  lenvnas  1  and  3  that  the  (m+l)-st  ratio  of 
the  full  structure  will  be  the  minimum  ratio  when  none  of  the  re¬ 
duced  ratios  are  candidates  and  the  problem  is  bounded.  Otherwise, 


from  lemmas  2  and  3  It  can  be  ascertained  that  only  ratios  which 
are  candlcates  In  both  the  reduced  and  full  structures  will  provide 
the  mlnlmums  for  the  respective  structures.  Furthermore,  lemma  4 
guarantees  that  these  mlnlmwns  will  occur  at  the  same  position  In 
the  reduced  and  full  structures. 
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4.3  Multiple  pivots 

The  general  Idea  of  a  multiple  pivot  Is  that  when  a  variable, 
say  "s,  enters  the  basis  In  place  of  a  current  basic  variable,  say  *p, 
the  Increase  In  the  objective  function  value,  X,  may  be  sufficient  to 

A 

cause  the  complement  of  *p  (call  It  *p)  to  become  eligible  to  enter 
the  basis.  The  word  complement  will  be  used  In  the  sense  that  the 
complement  of  w'  Is  *"  and  vice  versa.  (When  *  becomes  a  candidate 
In  this  dual  problem,  this  means  the  corresponding  constraint  In  the 
primal  problem  (5),  which  was  binding  at  one  bound  becomes  Infeasible 

A 

at  the  opposite  bound.)  Since,  for  this  case,  *p  would  become  a  can¬ 
didate  to  enter  at  the  next  Iteration  If  *s  enters  now,  the  algorithm 
brings  *p  Into  the  basis  Immediately  In  place  of  wp  ,  and  Is  still 
a  candidate  to  enter  in  place  of  some  other  basic  variable.  This 
results  In  a  computational  saving,  since  the  representations  of  *p 

and  »  are  Identical  in  terms  of  the  reduced  basis  B,  thus  no  basis 
P 

change  occurs  for  the  reduced  structure. 

For  this  discussion.  It  Is  assumed  that  has  been  selected 
to  enter  the  basis  In  place  of  *p  (where  may  be  **  or  and 
similarly  for  *  ),  and  that  »  «  *. .  1  <  t  <  «  ,  the  t-th  basic 

P  p  MCI  — 

variable.  The  representation  for  it  In  the  full  structure  Is  a  unit 

P 

vector,  but  the  representation  for  Its  complement,  wp,  may  be  obtain¬ 
ed  in  a  manner  similar  to  that  used  to  find  the  representation  for 


*  ,  so  the  discussion  Is  abbreviated. 

m  * 


33 


The  results  of  the  multiplication  and  solving  for  a.p  and  L  are 

a.  ■  (P  -  RL)  , 

P 

L  •  sunp/si»ir  , 

sump  •  L  -  OP  , 

A 

p  ■  a.p  ,  the  representation  for  (and  «p)  in  terms  of 

the  reduced  basis  B  . 

t .  i1 

(-1  If  w  Is  «’ 

'  p  p 

Then,  a  partial  tableau  for  the  full  problem,  written  In  terms  of  the 


reduced  basis  8,  is  shown 

below. 

A 

Basic  2_ 

_E 

\  RHS 

\  Cp  .  » 

0 

‘s  ‘ 

h  •  "tL 

•  • 

0 

• 

• 

S1  -  RjV  -R^sumr 

•  • 

•  • 

•  • 

•  • 

w  •  w  P .  -  R.L 

k(t)  p  t  t 

•  • 

• 

1 

• 

• 

•  • 

st  •  V  -Rt/Sl*r 

•  • 

•  • 

•  • 

•»(..  %  •  V 

• 

0 

•  • 

s.-v  •vs'*r 

*,(■»!>  1  •  IS 

0 

V  •  1®-  1/swr 

(Note  that  P1  ■  0  ,  1  P  t  ,  and  Pt  •  1 

colunn  of  B  and  P  ■  B  * 

,  since  A  k(t)  Is  the  t-th 

Since  *shas  been  selected  to  enter  the  basis  In  place  of 
the  change  In  X,  6,  which  will  result  when  enters  the  basis  Is 
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l  •  |cs/(-Rt/sumr)/(St  -  RfcV)|  , 

A 

6  »  |c$/sums|  . 

This  movement  in  A  ,6  >  may  be  sufficient  to  cause  n  to  become  a 

P 

candidate  to  enter  the  basis.  The  minimum  change  in  X  ,  d*  ,  which 

A 

will  enable  w  to  become  a  candidate  may  be  shown  to  be 
P 

6*  -  2X  ((-Rt/sumr)/(Pt-Rtl))  , 

which,  with  a  few  sign  changes  and  substitutions,  may  be  rewritten 
6*  ■  -2xRt/(2Rt  ♦  tsumr)  ,  for  p  ^  n^  . 

Therefore, 

a.  if  6  <  6*  ,  this  implies  that  when  *  enters  the  basis, 

—  * 

A 

the  change  in  X  is  not  sufficient  to  cause  *  to  become 

P 

a  candidate  to  enter  the  basis.  Thus,  ti  replaces  *  In 

»  p 

the  basis  and  the  usual  update  operations  take  place. 

b.  if  j*  <  4  ,  this  implies  that  if  w  enters  the  basis  in 

place  of  n  ,  the  change  in  X  would  be  sufficient  to 
P 

cause  n  to  become  a  candidate  to  enter  the  basis  at  the 
p 

next  iteration.  Therefore,  it  enters  inmedlately  in 

P 

place  of  *  and  *  is  still  a  candidate  to  replace  some 
p  • 

other  variable. 

(The  foregoing  assumed  ?p  was  selected  to  leave  the  basis 
and  p  <_  nj  .  For  the  case  where  p  >  n^  the  changes  are: 

a.  in  the  tableau,  c  ■  CIL  -  (X  Instead  of  2X  ; 

P  P  P 

b.  in  the  computation  of  6*  ,  CUp  -  CLp  replaces  2X  .) 

This  multiple  pivot  strategy  will  not  be  employed  when  the 

leaving  variable  Is  tr.  ,  since  If  the  complement  of  is 

klm*l)  kiow-l) 


brought  Into  the  basis,  the  algorithm  finds  the  nonbaslc  variable 
corresponding  to  the  next  minimum  ratio,  and  then  replaces  this 
next  variable,  and  the  basis  Is  updated.  However,  the  work  associated 
with  finding  the  next  smallest  ratio  and  then  performing  a  pivot  Is 
more  than  for  the  case  where  the  complement  of  "k(m+1)  does  not  enter 
the  basis,  and  a  multiple  pivot  is  not  performed. 

The  multiple  pivot  strategy  may  save  considerable  computation. 

As  evidenced  by  the  results  of  the  computational  experience,  the 
occasions  for  multiple  pivots  occur  between  17%  and  42%  of  the  time, 
depending  on  the  problem  site. 
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4.4  Steps  for  the  Reduced  Structure 

In  this  section,  the  steps  of  the  algorithm  discussed  In 
prior  sections  are  summarized.  Initially  assume  that  a  basic 
feasible  solution  is  at  hand. 

a.  the  parameters  to  estimate  In  the  primal  problem  (5) 

are  the  0’s,  which  are  Initially  computed  by  0  ■  CgB  1  . 

b.  the  value  of  the  objective  function,  X  ,  may  be  initially 


computed  by 


X  •  Jc  /sums j  . 


The  initial  situation  is  that  a  reduced  basis  for  (6), 

consisting  of  m  variables  is  obtained.  The  full  structure 

has  m+1  basic  variables  and  the  (m+1)-st  variable, 

jr  ,  is  a  slack  variable.  Recall  that  the  variable 

k(»*l) 

*  is  to  be  treated  as  a  pseudo-basic  variable  in 

k(»+i) 

the  reduced  structure,  and  at  this  juncture,  the  variable 

to  be  called  *  in  the  reduced  structure  has  not 

k(»*l) 

been  selected.  On  the  basis  of  reduced  costs,  suppose 

w  is  determined  to  be  eligible  to  enter  the  basis. 

a 

Instead  of  replacing  a  current  basic  variable  in  the 

reduced  structure,  v*  will  become  the  variable  called 

ir  ,  and  X  must  be  adjusted  to  make  this  variable 

k(»n) 

"basic."  In  the  full  structure,  however,  the  entering 
variable  does  replace  the  basic  slack  (artificial) 
variable  In  the  (m+l)-st  position.  Since  initially  the 
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Step  1 


first  m  basic  variables  In  the  full  structure  equal  zero 
(and  equals  one),  either  nk(t)  or  \<t>  * 

1  <  t  <  m  ,  may  be  selected  to  be  basic.  By  proper 
choice  of  w‘  or  *“  as  basic  variables,  the  ratio 

K  \  t)  Klv/ 

for  the  slack  variable  may  be  forced  to  be  minimum,  so 
that  the  entering  irg  will  replace  the  slack  a* 

desired.  After  the  slack  variable  leaves  the  basis, 
the  iterative  process  begins. 

Compute  the  reduced  costs 

a.  Cj  -  \ci  -  BTH.j|-X  •  |hj|-X  ,  for  j  <  n} 

1.  if  hj  >  0  ,  Cj  is  the  reduced  cost  for 

2.  If  hj  <  0  ,  -Cj  is  the  reduced  cost  for  njj  , 


J  i  ni 


bt  Cj  *  CLj  "  ®  •  J  ^  # 


1.  If  Cj  >  0  ,  Is  a  candidate  to  enter  the  basis 


aT 

n'L 


c.  Cj  ■  CUj  -  B  H.j  ,  j  >  n 


1 


1.  If  Cj  <  0  ,  irj  is  a  candidate  to  enter  the  basis 
d.  determine 

max  (c<  for  c,  >  0  ;  c\  for  c)  >  0  ;  -c\  for  c\  <  0) 
jtNB'H  NB"*  J  J  J 

and  let  the  variable  associated  with  this  stipulation  be 

»  ,  the  variable  to  enter  the  basis.  If  Ci  £0  , 

c j  <_  0  ,  and  Cj  >_  0  ,  terminate  with  the  optimal  solution. 
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Step  2.  Compute  the  reduced  ratios  to  determine  the  leaving  variable. 
Compute: 

min  {Rj/Sj)  ,  1  »  l,2,...,m  , 

and  consider  Rt/$t  a  candidate  if  ?g  •  -signfR^)  *  sign  (St)  , 
and  f  0. 

There  are  two  cases  which  may  occur: 

a.  if  the  minimum  ratio  is  R./S*  ,  *  ,  ,  leaves  the  reduced 

t  t  kit) 

basis,  1  <  t  <  m  ; 

b.  if  none  of  the  ratios  Is  a  candidate,  the  entering 

variable  replaces  the  pseudo-basic  variable  «  ..  , 

Mim4) 

and  no  basis  change  occurs.  In  the  full  structure,  of 

course,  an  explicit  basis  change  occurs  as  replaces 

the  basic  variable  *  .  . 

k(m+l) 

Step  3.  Update  the  value  of  x  . 

Let  X  be  the  current  value  of  X  and  *  has  been  selected  as 
the  entering  variable.  There  are  three  cases  to  consider. 

a.  If  the  variable  leaving  the  basis  (of  either  the  reduced 
or  full  structure)  Is  *rk(t)  ,  1  <  t  <  a  ,  then 

X  *  T  ♦  |cs(-Rt)/(Stsumr  -  Rtsums)| 

b.  If  the  variable  being  replaced  is  , 

X  *  T  ♦  | c  /sums]  . 

c.  For  the  case  where  a  multiple  pivot  occurs,  the  process 
for  updating  X  is  more  complicated.  Suppose  the  com¬ 
plement  of  a  current  basic  variable  enters  the  basis, 

A 

say  *p  replaces  i  (which  in  the  full  structure  would  be 
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described  by  k(t)  ■  p,  1  <  t  <  m. 

a 

A 

l.  When  the  exchange  occurs  (*p  replacing  »p),  X  Is 

updateu  by  computing 

X  ♦  X  -  KRt/(2Rt  ♦  tsumr), 

(  2X  If  t  <  n, 
where  K  ■  { 

(CUt  -  CLt  If  t  >  nj 

and  cg  Is  updated  by 

c.-c,  -  o*|suns  -  (St/Rt)su«r| . 

2.  After  this  exchange  and  updating,  the  Iteration  Is 

still  not  complete  since  the  variable  ,  which  was 

to  have  replaced  w  ,  must  now  enter  the  basis  In 

p 

place  of  some  other  basic  variable.  To  effect  this, 

the  next  smallest  ratio  Is  determined  (*  had  the 

P 

mlnlmun  ratio  previously),  and  the  variable  associat¬ 
ed  with  this  next  smallest  ratio  Is  replaced  by  . 
Then  X  Is  further  updated  to  reflect  this  change. 
There  are  three  cases  to  consider. 

a)  If  *k(t)  ,  1  <  t  <  m  ,  Is  replaced  by  , 

X  -  X  ♦  |c$(-Rt)/(StStfnr  -  Rtsums)|  ; 

b)  If  replaces  the  pseudo-basic  variable  \(w4l, 
In  the  reduced  structure  (and  In  the  full 
structure  *  replaces  the  basic  variable  *.  , 

In  an  explicit  basis  change),  then 
X  *  X  ♦  |cs/sums|  ; 


c)  If  a  multiple  pivot  occurs  here  (within  a 

multiple  pivot),  the  process  as  described  above 
repeats. 

Step  3.  Update  the  values  of  6  ,  by  computing 
6  -  (Cg  ♦  MB"1  . 

Go  to  Step  1. 


S.Q  Computational  Results 

The  special-purpose  algorithm  for  obtaining  L„  norm 
estimates  of  the  parameters,  which  may  have  linear  restrictions, 
for  the  model  C  •  AT8  ♦  c  Is  compared  to  a  general-purpose  linear 
programming  code.  Our  algorithm  has  been  coded  In  FORTRAN  as  a 
callable  subroutine,  which  requires  an  (nj+2n2)-dimens1onal  array 
and  an  n  by  m  matrix  for  the  Input  data,  an  m  by  m  matrix  for  the 
LU  decomposition,  and  seven  m-dlmenslona!  arrays. 

An  LU  decomposition  procedure  Is  used  to  solve  the  system 
of  equations,  and  since  much  of  the  original  data  are  preserved, 
the  Incidence  of  round-off  error  Is  diminished.  All  runs  were 
performed  on  a  CDC  6600  with  a  sixty  bit  word,  the  tolerance 
value  for  zero  was  set  at  l.E-8,  and  the  runs  were  made  within  a 
few  minutes  of  each  other,  so  the  machine  load  was  approximately 
the  same.  The  reported  times  were  from  using  the  RUN  compiler, 
although  the  algorithms  were  tested  using  the  FTN  and  MNF  compilers 
as  well.  The  IKSl  Library  was  utilized  to  generate  the  (nj*2n2)- 
dimenslonai  array  for  the  dependent  variable  C,  and  an  m  by  n 
matrix  A,  containing  data  which  were  randomly  drawn  from  a  uniform 
distribution.  The  results  which  are  summarized  In  table  1  Indicate 
our  code  (LFCON)  Is  ten  to  twenty  times  faster  than  the  general- 
purpose  code  LP6600. 


42 


TABLE  1:  A  summary  of  computational  testing  using  our  special-pur¬ 
pose  dual  algorithm  (IFCON)  and  LP6600  is  presented.  The  reported 
times  are  in  milliseconds,  using  a  CDC  6600  computer. 


Problem 

Size 

m.n.nf* 

Execution  Time 
(milliseconds) 
LFCON  LP6600 

PP'9 

Number  of 
Iterations 

LFCON  LP6600 

Objective 

function 

A 

2,  50.  1 

15 

326 

21.7  \  \ 

3 

8 

.7629 

2.  50.  2 

16 

324 

20.3  ■ 

2 

8 

.9234 

2.100,  2 

28 

557 

19.9 

2 

7 

.6772 

2,200,  2 

47 

1022 

21.7 

3 

7 

.8395 

2,300,  2 

96 

1889 

19.7 

6 

11 

2.2014 

5,  20,  2 

16 

282 

17.6 

3 

11 

.8092 

5,300,  5 

227 

3949 

17.4 

10 

20 

.8485 

5,500,  5 

303 

5668 

18.7 

10 

17 

1.6438 

10,200,  2 

515 

6309 

12.3 

20 

34 

.5635 

10,200,10 

510 

7187 

14.1 

21 

38 

.7214 

10,300,1 0 

689 

7842 

11.4 

22 

28 

.8317 

10,500,10 

1078 

16415 

15.2 

22 

37 

.7860 

10.500,15 

1373 

18188 

13.3 

33 

41 

.8145 

15,200,  5 

1572 

12711 

8.1 

38 

51 

.6240 

20,200,  2 

2881 

21927 

7.6 

41 

70 

.4841 

20,300,10 

3146 

32781 

10.4 

40 

71 

.8663 

20,400,  5 

4329 

ONR 

46 

ONR 

.5646 

20,500,  2 

6580 

ONR 

66 

ONR 

.5465 

*m  is  the  mmber  of  parameters  being  estimated 
n  is  the  nunber  of  variable  interval  constraints 
nf  is  the  number  of  fixed  interval  constraints 
ONR  means  the  problem  did  not  run 
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Conclusions: 

In  this  paper  we  have  presented  an  algorithm  and  FORTRAN 
code  for  determining  l„  norm  parameter  estimates  for  the  curve 
fitting  model  C  ■  ATa  ♦  c  ,  when  there  are  linear  restrictions 
on  the  parameters.  This  algorithm  is  a  special-purpose  linear 
programming  dual  method,  and  the  code  Is  shown  to  be  ten  to 
twenty  times  faster  than  a  general-purpose  linear  programming 
code  (LP6600). 

We  have  Indicated  how  to  partition  the  A  matrix  to 
form  a  compact  or  reduced  basis,  and  how  the  usual  simplex 
tableau  entries  may  be  written  In  terms  of  this  reduced  basis. 
Special  ratios  are  developed  for  determining  the  vector  to 
leave  the  basis,  which  are  of  a  slmpller  structure  than  the 
usual  simplex  ratios.  A  multiple  pivot  Is  developed,  where 
movement  Is  made  to  a  new  extreme  point  solution  without 
requiring  the  usual  basis  update  confutations. 
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